On the Equivalence Between a Minimal Codomain Cardinality Riesz Basis Construction, a System of Hadamard–Sylvester Operators, and a Class of Sparse, Binary Optimization Problems

Piecewise, low-order polynomial, Riesz basis families are constructed such that they share the same coefficient functionals of smoother, orthonormal bases in a localized indexing subset. It is shown that a minimal cardinality basis codomain can be realized by inducing sparsity, via l1 regularization, in the distributional derivatives of the basis functions and that the optimal construction can be found numerically by constrained binary optimization over a suitably large dictionary. Furthermore, it is shown that a subset of these solutions are equivalent to a specific, constrained analytical solution, derived via Sylvester-type Hadamard operators

Enhanced B-Wavelets via Mixed, Composite Packets

A modified B-wavelet construction with enhanced filter characteristics is considered. The design comprises a superposition of tessellated, integer dilated, ‘sister’ wavelet functions. We here propose a cascaded filter-bank realisation of this wavelet family together with some notable extensions. We prove that modifications of low-order members exist in the multiresolution subspace spanned by the half-translates of the original wavelets and hence that the resulting modified wavelet coefficients can be computed as convolutions of the undecimated original wavelet coefficients. Finite impulse response filters are thus designed and incorporated into a B-wavelet packet architecture such that the mainlobe-to-sidelobe ratio of the resulting wavelet filter characteristic is improved. This is achieved by designing the filters so that zeros are introduced near to the maxima of the harmonics. It is shown that the numbers of zeros can be balanced with the length of the corresponding filters by controlling the ‘modification order’. Several constructions are presented. We prove that two such constructions satisfy the perfect reconstruction property for all orders. The resulting modified wavelets preserve many of the properties of the original B-wavelets such as differentiability, number of vanishing moments, symmetry, anti-symmetry, finite support, and the existence of a closed form expression

Piecewise parameterised Markov random fields for semi-local Hurst estimation

Semi-local Hurst estimation is considered by incorporating a Markov random field model to constrain a wavelet-based pointwise Hurst estimator. This results in an estimator which is able to exploit the spatial regularities of a piecewise parametric varying Hurst parameter. The pointwise estimates are jointly inferred along with the parametric form of the underlying Hurst function which characterises how the Hurst parameter varies deterministically over the spatial support of the data. Unlike recent Hurst regularistion methods, the proposed approach is flexible in that arbitrary parametric forms can be considered and is extensible in as much as the associated gradient descent algorithm can accommodate a broad class of distributional assumptions without any significant modifications. The potential benefits of the approach are illustrated with simulations of various first-order polynomial forms

Sparse temporal difference learning via alternating direction method of multipliers

Recent work in off-line Reinforcement Learning has focused on efficient algorithms to incorporate feature selection, via 1-regularization, into the Bellman operator fixed-point estimators. These developments now mean that over-fitting can be avoided when the number of samples is small compared to the number of features. However, it remains unclear whether existing algorithms have the ability to offer good approximations for the task of policy evaluation and improvement. In this paper, we propose a new algorithm for approximating the fixed-point based on the Alternating Direction Method of Multipliers (ADMM). We demonstrate, with experimental results, that the proposed algorithm is more stable for policy iteration compared to prior work. Furthermore, we also derive a theoretical result that states the proposed algorithm obtains a solution which satisfies the optimality conditions for the fixed-point problem

M-estimate robust PCA for seismic noise attenuation

The robust principal component analysis (PCA) method has shown very promising results in seismic ambient noise attenuation when dealing with outliers in the data. However, the model assumes a general Gaussian distribution plus sparse outliers for the noise. In seismic data however, the noise standard variation could vary from one place to another leading to a more heavy-tailed noise distribution. In this paper, we present a new method which solves a convex minimisation problem of the robust PCA method with an M-estimate penalty function. Our empirical results show that the proposed method can outperform the robust PCA method

Multi-scale Sparse Coding With Anomaly Detection And Classification

We here place a recent joint anomaly detection and classification approach based on sparse error coding methodology into multi-scale wavelet basis framework. The model is extended to incorporate an overcomplete wavelet basis into the dictionary matrix whereupon anomalies at specified multiple levels of scale are afforded equal importance. This enables, for example, subtle transient anomalies at finer scales to be detected which would otherwise be drowned out by coarser details and missed by the standard sparse coding techniques. Anomaly detection in power networks provides a motivating application and tests on a real-world data set corroborates the efficacy of the proposed model

Generalised M-Lasso for robust, spatially regularised hurst estimation

A generalised Lasso iteratively reweighted scheme is here introduced to perform spatially regularised Hurst estimation on semi-local, weakly self-similar processes. This is extended further to the robust, heavy-tailed case whereupon the generalised M-Lasso is proposed. The design successfully incorporates both a spatial derivative in the generalised Lasso regulariser operator and a weight matrix formulated in the wavelet domain. The result simultaneously spatially smooths the Hurst estimates and downweights outliers. Experiments using a Hampel score function confirm that the method yields superior Hurst estimates in the presence of strong outliers. Moreover, it is shown that the inferred weight matrix can be used to perform wavelet shrinkage and denoise fractional Brownian surfaces in the presence of strong, localised, band-limited noise

Semi-local scaling exponent estimation with box-penalty constraints and total-variation regularisation

We here establish and exploit the result that 2-D isotropic self-similar fields beget quasi-decorrelated wavelet coefficients and that the resulting localised log sample second moment statistic is asymptotically normal. This leads to the development of a semi-local scaling exponent estimation framework with optimally modified weights. Furthermore, recent interest in penalty methods for least squares problems and generalised Lasso for scaling exponent estimation inspires the simultaneous incorporation of both bounding box constraints and total variation smoothing into an iteratively reweighted least-squares estimator framework. Numerical results on fractional Brownian fields with global and piecewise constant, semi-local Hurst parameters illustrate the benefits of the new estimators

The time-varying dependency patterns of NetFlow statistics

We investigate where and how key dependency structure between measures of network activity change throughout the course of daily activity. Our approach to data-mining is probabilistic in nature, we formulate the identification of dependency patterns as a regularised statistical estimation problem. The resulting model can be interpreted as a set of time-varying graphs and provides a useful visual interpretation of network activity. We believe this is the first application of dynamic graphical modelling to network traffic of this kind. Investigations are performed on 9 days of real-world network traffic across a subset of IP's. We demonstrate that dependency between features may change across time and discuss how these change at an intra and inter-day level. Such variation in feature dependency may have important consequences for the design and implementation of probabilistic intrusion detection systems

Improving hyperspectral band selection by constructing an estimated reference map

We investigate band selection for hyperspectral image classification. Mutual information (MI) measures the statistical dependence between two random variables. By modeling the reference map as one of the two random variables, MI can, therefore, be used to select the bands that are more useful for image classification. A new method is proposed to estimate the MI using an optimally constructed reference map, reducing reliance on ground-truth information. To reduce the interferences from noise and clutters, the reference map is constructed by averaging a subset of spectral bands that are chosen with the best capability to approximate the ground truth. To automatically find these bands, we develop a searching strategy consisting of differentiable MI, gradient ascending algorithm, and random-start optimization. Experiments on AVIRIS 92AV3C dataset and Pavia University scene dataset show that the proposed method outperformed the benchmark methods. In AVIRIS 92AV3C dataset, up to 55% of bands can be removed without significant loss of classification accuracy, compared to the 40% from that using the reference map accompanied with the dataset. Meanwhile, its performance is much more robust to accuracy degradation when bands are cut off beyond 60%, revealing a better agreement in the MI calculation. In Pavia University scene dataset, using 45 bands achieved 86.18% classification accuracy, which is only 1.5% lower than that using all the 103 bands